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DETAILED ACTION 

1 . This office action is responsive to communication filed on August 20, 2008. 
Claims 16 and 17 are pending in the application and have been examined by the 
Examiner. 

Continued Examination Under 37 CFR 1.114 

2. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on August 
20, 2008 has been entered. 

Response to Arguments 

3. Applicant's arguments filed August 20, 2008 have been fully considered but they 
are not persuasive. 

4. Consider claim 16, Applicant argues the following: In summary, Nakamura fails 
to teach the inventive aspects as indicated above, i.e., (1) a color space conversion 
operation for the first RAW data is performed in accordance with the start of reading the 
second RAW data, (2) the color space conversion for the first RAW data is performed at 
the same time (i.e., in parallel) as the white balance integral processing of the second 
RAW data (i.e., while the second RAW data are read from the image sensing element), 
(3) the display device displays the object image of the third RAW data after the color 
space conversion of the first RAW data and the integral processing of the second PAW 
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data, (4) the white balance calculation device calculates the white balance coefficient of 
the second RAW data while the display device displays the object image of the third 
RAW data (i.e., after the parallel processing of the color space conversion of the first 
RAW data and the white balance integral processing of the second RAW data), and (5) 
the third RAW data are stored in the first area where the first RAW data used to be 
stored. 

5. The Examiner respectfully disagrees. Regarding the first argument, the 
Examiner agrees that Nakamura does not explicitly teach that a color space conversion 
operation for the first RAW data (i.e. IMAGE PROCESSING 1 , figure 8) is performed in 
accordance with the start of reading the second RAW data (READOUT 2, figure 8). 
However, this is remedied by Anderson, who teaches of a parallel operation of reading 
out second RAW data while processing first RAW data, as previously and currently 
discussed in the body of the rejection of claim 16. 

6. Regarding the second argument, the Examiner agrees that Nakamura does not 
explicitly teach that the color space conversion for the first RAW data (i.e. IMAGE 
PROCESSING 1 , figure 8) is performed at the same time (i.e., in parallel) as the white 
balance integral processing of the second RAW data (i.e., while the second RAW data 
are read from the image sensing element, READOUT 2, figure 8). However, this is 
remedied by Anderson, who teaches of a parallel operation of reading out second RAW 
data while processing first RAW data, as previously and currently discussed in the body 
of the rejection of claim 16. 
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7. Regarding the third argument, the Examiner has been unable to find support for 
the amended limitation that the display device displays the object image of the third 
RAW data after the color space conversion of the first RAW data and the integral 
processing of the second RAW data in the original disclosure. This is discussed in the 
35 U.S.C. 112 rejection below. 

8. Regarding the fourth argument, the Examiner has been unable to find support for 
the amended limitation that the white balance calculation device calculates the white 
balance coefficient of the second RAW data while the display device displays the 
object image of the third RAW data (i.e., after the parallel processing of the color 
space conversion of the first RAW data and the white balance integral processing of the 
second RAW data) in the original disclosure. This is discussed in the 35 U.S.C. 112 
rejection below. 

9. Regarding the fifth argument, the fact that Nakamura may not teach that the third 
RAW data are stored in the first area where the first RAW data used to be stored is a 
moot point, as Nakamura is not relied upon to satisfy the limitation of first and second 
areas of memory. This is remedied by Anderson. Furthermore, Anderson teaches first 
and second areas of memory (see figure 4B), which alternately store image data such 
that a first area of memory (Input Buffer A) stores first, third, etc. RAW data and a 
second area (Input Buffer B) stores second, fourth, etc. Raw data, as previously and 
currently discussed in the body of the rejection of claim 16. 
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1 0. With respect to the Anderson reference, Applicant argues that there is simply 
nothing in Anderson including the cited portions that teaches that the color conversion 
of the first RAW data is performed with the start of reading the second RAW data. 

1 1 . The Examiner respectfully disagrees. First off, Nakamura teaches that color 
conversion is performed during the image processing of the first RAW data (PC, figure 
8, column 7, lines 24-33). Anderson teaches that image processing of the first RAW 
data is performed with the start of reading of the second RAW data(See figure 4a, 
column 4, line 59 through column 6, line 3, column 6, lines 38-56, column 8, line 59 
through column 9, line 8. Anderson teaches, "Referring again to FIG. 4B, the ping-pong 
buffers are utilized during live view mode as follows. While input buffer A is filled with 
image data, the data from input buffer B is processed and transmitted to frame buffer B. 
At the same time, previously processed data in frame buffer A is output to the LCD 
screen 402 for display. While input buffer B is filled with image data, the data from input 
buffer A is processed and transmitted to frame buffer A. At the same time, previously 
processed data in frame buffer B is output to the LCD screen 402 for display." As one 
buffer (i.e. the second area) is filled with raw image data (i.e. the start of reading of the 
second RAW data from the image sensing element), the other buffer (i.e. the first area) 
is emptied and processed (i.e. first RAW data is readout from the first area), which 
processing involves color space conversion (See 612, figure 7, column 8, line 59 
through column 9, line 8). Anderson teaches that the input buffers A and B alternate 
between an input cycle and a processing cycle, column 6, lines 8-10.). 
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12. Finally, Applicant argues that the apparatus disclosed in Anderson has input 
buffers for n pieces of RAW data that correspond to a number of input RAW data, and 
does not have a structure in which two areas are alternately used. Furthermore, 
Anderson is silent in disclosing the kind of image processing performed during the live 
view operation, as required by the present invention. 

13. The Examiner respectfully disagrees. Anderson teaches that each buffer holds 
one image, and that the buffers are alternately filled and emptied (column 6, lines 37- 
56). Anderson further explicitly teaches the kind of image processing performed during 
live view operation in at least column 6, lines 1 9-32. 

14. Therefore, the rejection is maintained by the Examiner. 

Claim Objections 

1 5. Claim 1 7 is objected to because of the following informalities: Lack of clarity and 
precision. 

Claims 17 recites, "the apparatus according to claim 17". Upon further 
examination, it appears that claim 17 is meant to depend from claim 16. Therefore, the 
Examiner will interpret claim 17 to read, "the apparatus according to claim 16". 
Appropriate correction is required. 

Claim Rejections - 35 USC §112 

16. The following is a quotation of the first paragraph of 35 U.S. C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 
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17. Claim 16 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor(s), at the time the application was filed, had 
possession of the claimed invention. 

Claim 16 recites, "and said display device displays the object image of the 

third RAW data after the color space conversion processing for the first RAW data and 
the integral processing for the second RAW data, and said white balance calculation 
device calculates the white balance coefficient of the second RAW data while said 
display device displays the object image of the third RAW data after the parallel 
processing of the first and second RAW data, and said memory stores the third RAW 
data in the first area in which the color space conversion of the first RAW data by said 
image processing device is finished". 

Applicant recites, on page 20, lines 5-7 of the specification, "Photographing 
operation of the electronic camera for the third and subsequent images is also the same 
as that for the second image." Figure 2 shows the steps of the photographing 
operation. First (S200), the electronic viewfinder (i.e. display) is stopped, next there is 
an exposure of the image sensor (S201), and finally a white-balance integration is 
performed (S204 or S205). Applicant recites, on page 13, lines 13-16, that an image is 
obtained in step 201 by driving the shutter (2). This is subsequent to the turning off of 
the display in step 200. Therefore, because the operation for obtaining RAW data to be 
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recorded is performed during an exposure period in which the display is turned off, the 
data stored in memory is not the same as data displayed as the object image. The 
Examiner has been unable to find anywhere in the original disclosure reciting that the 
display displays the RAW data, and that the said RAW data from the display is 
subsequently stored in one of the areas of memory. 

Therefore, the Examiner will interpret claim 16 to read, "and said display device 
displays the object image after the color space conversion processing for the first RAW 
data and the integral processing for the second RAW data, and said white balance 
calculation device calculates the white balance coefficient of the second RAW data 
while said display device displays the object image after the parallel processing of the 
first and second RAW data, and said memory stores the third RAW data in the first area 
in which the color space conversion of the first RAW data by said image processing 
device is finished". 

Claim Rejections - 35 USC § 103 

18. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

19. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 
al. (US 5,619,347). 
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Consider claim 16, Nakamura et al. teaches: 

An image sensing apparatus ("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device ("CCD", 303, figure 4) which outputs image data 
obtained by an image sensing element as RAW data (column 3, lines 50-58); 

a memory ("DRAM", 232, figure 4) which has a first area for temporally storing 
first RAW data obtained in a first image sensing operation of said image sensing device 
and for temporally storing second RAW data obtained in a second image sensing 
operation next to the first image sensing operation of said image sensing device (See 
figure 7, column 7, lines 10-49. First raw image data is written into the DRAM (232) 
over channel 1 . During a subsequent image capture operation, this first raw data is 
read out of DRAM (232) over channel 2 for image processing, as second image data is 
written into the DRAM (232) over channel 1 .); 

a white balance integration device (21 1a, figure 6A) which integrates at least one 
of the first and second RAW data for a white balance processing (The white balance 
integration device (21 1 a) is part of the image signal processor (21 1 , figure 4). The 
RAW image data is subjected to white balance processing, and then stored into the 
DRAM (232), column 6, lines 21-26.); 

an image processing device (21 1 , figure 4) which performs image processing of 
the first and second RAW data readout from said memory (The image processing 
device (21 1) performs processing such as color space conversion on the RAW data 
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readout from memory (232), column 4, lines 8-10, column 6, lines 26-36, column 7, lines 
24-33.), 

a display device ("EVF", 20, or "LCD", 10, figure 4) which displays an object 
image during imaging on the image sensing element (The display acts as a "live view 
display" (i.e. an object image is displayed during imaging), column 3, lines 16-23.); and 

a control device ("main CPU", 21, figure 4) which controls said memory (232), 
said white balance integration device (21 1a), said image processing device (21 1), and 
said display device (20, 10, column 4, lines 1-27. The main CPU (21) comprises the 
image processing device (21 1 ) which contains the white balance integration device 
(21 1a), a bus controller (218) for controlling the memory (232), and a video encoder 
(213) which supplies analog image signals to the display.), 

wherein, in case said image sensing device outputs third RAW data obtained in a 
third image sensing operation next to the first and second image sensing operations 
(See figure 7. Steps 1 , 8 and 10-12 comprise a continuous loop. Because of this, it is 
clear that third RAW data, fourth RAW data, etc. can be output by the image sensing 
device without altering the camera operation.), said control device (21) controls so that, 
said image processing device (21 1 ) processes a color space conversion for the first 
RAW data readout from said memory (232) in accordance with start of 
exposure/storage of the second RAW data from the image sensing element (303) in the 
second image sensing operation (See figure 8, column 7, lines 24-33. A frame of raw 
data obtained by an immediately preceding image (i.e. first image data) is read out of 
DRAM (232) and subjected to color space conversion while second image data is 
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obtained through exposure and storage of the CCD.), and said display device ("LCD", 
10) displays the object image (See "Live View Display" on the right side of figure 8.) 
after the color space conversion processing for the first RAW data (See "Pc", figure 8) 
and the integral processing (See Readout 2, "Pe", figure 8) for the second RAW data 
(See column 7, lines 24-49. The LCD exhibits a "Live View Display" after the color 
space conversion "Pc" and integral processing "Pe".). 

However, Nakamura et al. does not explicitly teach that said memory (232) has a 
second area for storing the second RAW image data, each of the first and second areas 
of the memory storing the RAW data from the first and second operations alternately, or 
that said memory stores the third RAW data in the first area in which the first RAW data 
after processing of the first RAW data by said image processing device is finished. 
Nakamura et al. further does not explicitly teach that the processing device processes a 
color space conversion for the first RAW data readout from said first area in accordance 
with start of reading the second RAW data from the image sensing element. 

Anderson is similar to Nakamura et al. in that Anderson teaches of a camera 
(figures 1-3) with a memory (figure 4a). Anderson also similarly teaches of reading out 
raw image data from an image sensor (1 14, figure 1 , column 5, lines 59-64), storing the 
data in a memory (530, column 5, line 59 through column 6, line 3), and subsequently 
performing color space conversion on the image data (column 8, line 59 through column 
9, line 7). 

However, in addition to Nakamura et al., Anderson teaches that said memory 
(figures 4a and 4b) has a second area (Input buffer, 2, B) for storing the second RAW 
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image data, each of the first and second areas of memory stores the RAW data from 
the first and second operations alternately, that said memory stores the third RAW data 
in the first area in which the color space conversion of the first RAW data by said 
image processing device is finished, and that the processing device processes a color 
space conversion for the first RAW data readout from said first area in accordance with 
start of reading the second RAW data from the image sensing element (See figure 4a, 
column 4, line 59 through column 6, line 3, column 6, lines 38-56, column 8, line 59 
through column 9, line 8. Anderson teaches, "Referring again to FIG. 4B, the ping- 
pong buffers are utilized during live view mode as follows. While input buffer A is filled 
with image data, the data from input buffer B is processed and transmitted to frame 
buffer B. At the same time, previously processed data in frame buffer A is output to the 
LCD screen 402 for display. While input buffer B is filled with image data, the data 
from input buffer A is processed and transmitted to frame buffer A. At the same time, 
previously processed data in frame buffer B is output to the LCD screen 402 for 
display." As one buffer (i.e. the second area) is filled with raw image data (i.e. the start 
of reading of the second RAW data from the image sensing element), the other buffer 
(i.e. the first area) is emptied and processed (i.e. first RAW data is readout from the 
first area), which processing involves color space conversion (See 612, figure 7, 
column 8, line 59 through column 9, line 8). Anderson teaches that the input buffers A 
and B alternate between an input cycle and a processing cycle, column 6, lines 8-10. 
Therefore, buffer A is an input buffer during a first phase, an output buffer during a 
second phase when second RAW data is written into buffer B, and again an input 
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buffer during a third phase when third RAW data is output from the sensor an into 
memory. Thus, the third image data is stored in the same area (i.e. buffer A) as the 
first image data during the third phase, after the first image data is processed during 
the second phase.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use ping-pong buffers as taught by Anderson in the 
camera taught by Nakamura et al. to read out raw image data from the image sensor 
concurrent with the processing color space conversion of previous image data, for the 
benefit of improving the display speed of the digital camera and preventing the tearing 
of the image on the display (Anderson, column 5, line 65 through column 6, line 3.). 

Nakamura et al. alone does not explicitly teach that the integral processing 
for the second RAW data by said white balance integration device and the color 
space conversion for first RAW data by said image processing device processes 
are performed in parallel during reading of the second RAW data from the image 
sensing element. However, because the white balance integration taught by 
Nakamura et al. is performed during the raw data writing ("Pe"), and thus in 
parallel with the readout of the second image data (see figure 8, column 7, lines 
41-49), the combination of Nakamura et al. and Anderson teaches that that 
integral processing of the second image data and the color space conversion of 
the first image data are performed in parallel. This is because Anderson modifies 
Nakamura et al. such that the image processing of the first image data (which 
includes color space conversion) readout from the first area of memory takes 
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place concurrently with the readout of second RAW image data from the image 
sensor (which includes integration) and into the second area of memory, as 
discussed above. 

However, the combination of Nakamura et al. and Anderson does not explicitly 
teach a white balance calculation device which calculates a white balance coefficient on 
the basis of the integration result by the white balance calculation, or that the image 
processing is performed in accordance with the white balance coefficient calculated by 
said white balance calculation device. 

Taniguchi et al. is similar to Nakamura et al. in that Taniguchi et al. teaches 
performing white balance (column 1 , lines 9-15) on image data stored in a picture 
memory (1 2, figure 1 , column 9, lines 42-60). Taniguchi et al. also similarly teaches of a 
white balance integration device (15, 16, figure 1, column 9, line 63 through column 10, 
line 8). 

However, in addition to the teachings of Nakamura et al. and Anderson, 
Taniguchi et al. teaches of a white balance calculation device ("white balance coefficient 
calculating unit", 22, figure 1) which calculates a white balance coefficient on the basis 
of the integration result by the white balance calculation (See figure 1, column 10, lines 
45-60. The white balance calculation device (22) calculates a white balance coefficient 
according to a plurality of factors determined in units 17-21 of figure 1, based upon the 
white balance integration of the white balance integration device (15,16). See also 
column 10, lines 6-44.), and that the image processing is performed in accordance with 
the white balance coefficient calculated by said white balance calculation device (See 
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14, figure 1, column 9, lines 50-60, column 10, lines 56-60. Image processing is 
performed by the white balance adjusting unit (14) based upon the white balance 
coefficient calculated by the white balance coefficient calculating unit (22).). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have the image processing device taught by the 
combination of Nakamura et al. and Anderson comprise a white balance calculation 
device and perform white balance processing based upon a calculated white balance 
coefficient as taught by Taniguchi et al. for the benefit of performing a sufficient degree 
of white balance adjustment appropriate to a colored picture without any erroneous 
adjustment or adverse influence due to a high chromaticity region of the colored picture 
(Taniguchi et al., column 2, lines 40-50). 

The combination of Nakamura et al., Anderson, and Taniguchi et al. teaches that 
said white balance calculation device (taught by Taniguchi et al.) calculates the white 
balance coefficient for the second RAW data (see above rationale) while said display 
device displays the object image after the parallel processing of the first and second 
RAW data. Anderson teaches that parallel processing is performed by utilizing two 
buffers, column 6, lines 47-56. Anderson teaches that the image data from one buffer is 
displayed on the LCD (402) as the image data from the other buffer is processed, 
column 6, lines 47-56. Therefore, an object image is continuously displayed on the 
display, including any time when a white balance coefficient is being calculated. 
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20. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 
al. (US 5,61 9,347) as applied to claim 1 6 above, and further in view of Sasaki (US 
6,961,085). 

Consider claim 17, and as applied to claim 16 above, the combination of 
Nakamura et al., Anderson, and Taniguchi et al. teaches that said display device 
displays the object image after said white balance calculation device calculates the 
white balance coefficient (See claim 16 rationale. Anderson teaches of a continuous 
display.). However, the combination of Nakamura et al., Anderson, and Taniguchi et al. 
does not explicitly teach a defect correction device. 

Sasaki is similar to Nakamura et al. in that image data is collected from the 
image sensor (12, figure 1 ), preliminary processing is performed to yield first image data 
(see step 41 , figure 7) which is written into a buffer memory (26a, figure 7). Sasaki also 
similarly teaches that the first image data is read from the buffer memory (step 42, 
figure 7) for additional processing (see column 10, lines 1-57). 

However, in addition to the combined teachings of Nakamura et al., Anderson, 
and Taniguchi et al., Sasaki teaches that the apparatus further comprises a defect 
correction device which corrects a defective pixel portion of image data when the image 
sensing element has a defective pixel (See column 10, lines 21-57. Sasaki teaches that 
the locations of defective pixels are stored in memory, and when reading the data 
output from the buffer (26a), the defective pixels are corrected.), and that a control 
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device controls said defect correction device in such a way that said defect correction 
device corrects a defective pixel portion of the image data (See column 10, lines 29-31, 
lines 37-41.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Sasaki correct defective pixels in the image processing device 
during the display of the object image as taught by the combination of Nakamura et al., 
Anderson and Taniguchi et al. for the benefit of keeping the influence of a defective 
pixel to a minimum and preserving a high-definition image (Sasaki, column 3, lines 1-5, 
lines 12-16). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALBERT H. CUTLER whose telephone number is 
(571)270-1460. The examiner can normally be reached on Mon-Thu (9:00-5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc-Yen Vu can be reached on (571) 272-7320. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

AC 

09/11/2008 



/Ngoc-Yen T. VU/ 
Supervisory Patent Examiner, Art Unit 2622 



